Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants

Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques

Video, as a key driver in the global explosion of digital information, can create tremendous benefits for human society. Governments and enterprises are deploying innumerable cameras for a variety of applications, e.g., law enforcement, emergency management, traffic control, and security surveillance, all facilitated by video analytics (VA). This trend is spurred by the rapid advancement of deep learning (DL), which enables more precise models for object classification, detection, and tracking. Meanwhile, with the proliferation of Internet-connected devices, massive amounts of data are generated daily, overwhelming the cloud. Edge computing, an emerging paradigm that moves workloads and services from the network core to the network edge, has been widely recognized as a promising solution. The resulting new intersection, edge video analytics (EVA), begins to attract widespread attention. Nevertheless, only a few loosely-related surveys exist on this topic. The basic concepts of EVA (e.g., definition, architectures) were not fully elucidated due to the rapid development of this domain. To fill these gaps, we provide a comprehensive survey of the recent efforts on EVA. In this paper, we first review the fundamentals of edge computing, followed by an overview of VA. The EVA system and its enabling techniques are discussed next. In addition, we introduce prevalent frameworks and datasets to aid future researchers in the development of EVA systems. Finally, we discuss existing challenges and foresee future research directions. We believe this survey will help readers comprehend the relationship between VA and edge computing, and spark new ideas on EVA.Comment: 31 pages, 13 figure

Mechanical Characterizations of Oxidizing Steel Slag Soil and Application

In this paper, the mechanical properties and engineering applicationof electric furnace (EAF) slag mixed soil are investigated.The samples of steel slag are taken from a steel manufacturingcompany in Huangshi, a city of China. The mixed soilwas firstly prepared by mixing the steel slag and clay mixturein different proportions. The optimal moisture content for mixingthe soil is investigated from the experiment through directshear test. Based on three axial compression tests, the optimumsteel slag ratio is determined. Finally, the mechanical propertiesof steel slag mixed soil are tested in a practical engineeringproblem through a numerical simulation. The steel slag mixedsoil is used to replace the original soil of the embankment andcompared with that of the original one. The comparison studyshows that the method proposed in this paper is simple andeffective. Moreover, from the practical problem analysis, theoptimal utilization of electric furnace slag can be achieved

Macro action selection with deep reinforcement learning in StarCraft

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also considered as a testbed for AI research, due to its enormous state space, hidden information, multi-agent collaboration and so on. Thanks to the annual AIIDE and CIG competitions, a growing number of bots are proposed and being continuously improved. However, a big gap still remains between the top bot and the professional human players. One vital reason is that current bots mainly rely on predefined rules to perform macro actions. These rules are not scalable and efficient enough to cope with the large but partially observed macro state space in SC. In this paper, we propose a DRL based framework to do macro action selection. Our framework combines the reinforcement learning approach Ape-X DQN with Long-Short-Term-Memory (LSTM) to improve the macro action selection in bot. We evaluate our bot, named as LastOrder, on the AIIDE 2017 StarCraft AI competition bots set. Our bot achieves overall 83% win-rate, outperforming 26 bots in total 28 entrants

Virtual k -Space Modulation Optical Microscopy

We report a novel superresolution microscopy approach for imaging fluorescence samples. The reported approach, termed virtual k-space modulation optical microscopy (VIKMOM), is able to improve the lateral resolution by a factor of 2, reduce the background level, improve the optical sectioning effect and correct for unknown optical aberrations. In the acquisition process of VIKMOM, we used a scanning confocal microscope setup with a 2D detector array to capture sample information at each scanned x-y position. In the recovery process of VIKMOM, we first modulated the captured data by virtual k-space coding and then employed a ptychography-inspired procedure to recover the sample information and correct for unknown optical aberrations. We demonstrated the performance of the reported approach by imaging fluorescent beads, fixed bovine pulmonary artery endothelial (BPAE) cells, and living human astrocytes (HA). As the VIKMOM approach is fully compatible with conventional confocal microscope setups, it may provide a turn-key solution for imaging biological samples with ∼100  nm lateral resolution, in two or three dimensions, with improved optical sectioning capabilities and aberration correcting.National Institutes of Health (U.S.) (9P41EB015871-26A1)National Institutes of Health (U.S.) (1R01HL121386-01A1

dRecQ4 Is Required for DNA Synthesis and Essential for Cell Proliferation in Drosophila

Background: The family of RecQ DNA helicases plays an important role in the maintenance of genomic integrity. Mutations in three of the five known RecQ family members in humans, BLM, WRN and RecQ4, lead to disorders that are characterized by predisposition to cancer and premature aging. Methodology/Principal Findings: To address the in vivo functions of Drosophila RecQ4 (dRecQ4), we generated mutant alleles of dRecQ4 using the targeted gene knock-out technique. Our data show that dRecQ4 mutants are homozygous lethal with defects in DNA replication, cell cycle progression and cell proliferation. Two sets of experiments suggest that dRecQ4 also plays a role in DNA double strand break repair. First, mutant animals exhibit sensitivity to gamma irradiation. Second, the efficiency of DsRed reconstitution via single strand annealing repair is significantly reduced in the dRecQ4 mutant animals. Rescue experiments further show that both the N-terminal domain and the helicase domain are essential to dRecQ4 function in vivo. The N-terminal domain is sufficient for the DNA repair function of dRecQ4. Conclusions/Significance: Together, our results show that dRecQ4 is an essential gene that plays an important role in no